Physically realistic computer simulation of medical procedures

ABSTRACT

An apparatus for interfacing the movement of a shaft with a computer includes a support, a gimbal mechanism having two degrees of freedom, and three electromechanical transducers. When a shaft is engaged with the gimbal mechanism, it can move with three degrees of freedom in a spherical coordinate space, where each degree of freedom is sensed by one of the three transducers. A fourth transducer can be used to sense rotation of the shaft around an axis.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.10/674,423, filed Oct. 1, 2003 now U.S. Pat. No. 7,215,326, which is acontinuation of U.S. application Ser. No. 09/996,487, filed Nov. 27,2001, now U.S. Pat. No. 6,654,000, which is a continuation of U.S.application Ser. No. 09/276,012, filed Mar. 25, 1999, now U.S. Pat. No.6,323,837, which is a continuation of application Ser. No. 08/833,502,filed Apr. 7, 1997 now U.S. Pat. No. 6,037,927, which is a continuationof application Ser. No. 08/275,120, filed Jul. 14, 1994, now U.S. Pat.No. 5,623,582.

BACKGROUND OF THE INVENTION

This invention relates generally to human/computer interface devices,and more particularly to computer input devices such as mice,trackballs, etc.

Virtual reality computer systems provide users with the illusion thatthey are part of a “virtual” environment. A virtual reality system willtypically include a personal computer or workstation, specializedvirtual reality software, and virtual reality I/O devices such as headmounted displays, pointer gloves, 3D pointers, etc.

For example, a virtual reality computer system can allow adoctor-trainee or other human operator or user to “manipulate” a scalpelor probe within a computer-simulated “body”, and thereby perform medicalprocedures on a virtual patient. In this instance, the I/O device istypically a 3D pointer, stylus, or the like. As the “scalpel” or “probe”moves within the body image displayed on the screen of the computersystem, results of such movement are updated and displayed so that theoperator can gain the experience of such a procedure without practicingon an actual human being or a cadaver.

For virtual reality systems to provide a realistic (and thereforeeffective) experience for the user, sensory feedback and manualinteraction should be as natural as possible. As virtual reality systemsbecome more powerful and as the number of potential applicationsincreases, there is a growing need for specific human/computer interfacedevices which allow users to interface with computer simulations withtools that realistically emulate the activities being represented withinthe virtual simulation. Such procedures as laparoscopic surgery,catheter insertion, and epidural analgesia should be realisticallysimulated with suitable human/computer interface devices if the doctoris to be properly trained.

While the state of the art in virtual simulation and medical imagingprovides a rich and realistic visual feedback, there is a great need fornew human/computer interface tools which allow users to perform naturalmanual interactions with the computer simulation. For medicalsimulation, there is a strong need to provide doctors with a realisticmechanism for performing the manual activities associated with medicalprocedures while allowing a computer to accurately keep track of theiractions.

There are number of devices that are commercially available forinterfacing a human with a computer for virtual reality simulations.There are, for example, such 2-dimensional input devices such as mice,trackballs, and digitizing tablets. However, 2-dimensional input devicestend to be awkward and inadequate to the task of interfacing with3-dimensional virtual reality simulations. In contrast, a 3-dimensionalhuman/computer interface tool sold under the trademark Immersion PROBE™is marketed by Immersion Human Interface Corporation of Palo Alto,Calif., and allows manual control in 3-dimensional virtual realitycomputer environments. A pen-like stylus allows for dexterous3-dimensional manipulation, and the position and orientation of thestylus is communicated to a host computer. The Immersion PROBE has sixdegrees of freedom which convey spatial coordinates (x, y, z) andorientation (role, pitch, yaw) of the stylus to the host computer.

While the Immersion PROBE is an excellent 3-dimensional interface tool,it may be inappropriate for certain virtual reality simulationapplications. For example, in some of the aforementioned medicalsimulations three or four degrees of freedom of a 3-dimensionalhuman/computer interface tool is sufficient and, often, more desirablethan five or six degrees of freedom because it more accurately mimicsthe real-life constraints of the actual medical procedure. Therefore, aless complex, more compact, lighter weight, lower inertia and lessexpensive alternative to six degree of freedom human/computer interfacetool is desirable for certain applications.

SUMMARY

The present invention provides a 3-dimensional human/computer interfacetool which is particularly well adapted to virtual reality simulationsystems that require fewer degrees of freedom, e.g. two, three, or fourdegrees of freedom. The present invention therefore tends to be lesscomplex, more compact, lighter weight, less expensive, more reliable andhave less inertia than 3-dimensional human/computer interface tools ofthe prior art having more degrees of freedom.

The present invention is directed to a method and apparatus forproviding an interface between a human and a computer. The human end ofthe interface is preferably a substantially cylindrical object such as ashaft of a surgeon's tool, a catheter, a wire, etc. Alternatively, itcan comprise a pool cue, a screw driver shaft, or any other elongatedobject that is manipulated in 3-dimensional space by a human operator.In certain embodiments of the present invention, the computer developssignals to provide force feedback to the object. For example, a twistingor resisting force can be imparted on the object to provide haptic orforce feedback of a medical procedure being performed in a virtualreality simulation.

An apparatus for interfacing with a electrical system includes asupport, a gimbal mechanism coupled to the support, and preferably threeelectromechanical transducers, although certain embodiments (e.g. foruse with catheters) may require only two electromechanical transducers.The gimbal mechanism has a base portion which is rotatably coupled tothe support to provide a first degree of freedom, and an objectreceiving portion rotatably coupled to the base portion to provide asecond degree of freedom. A first electromechanical transducer iscoupled between the support and the base portion, a secondelectromechanical transducer is coupled between the base portion and theobject receiving portion, and a third electromechanical transducer iscoupled between the object receiving portion and an intermediate portionof an elongated object that is at least partially disposed within theobject receiving portion. The third electromechanical transducer isassociated with a third degree of freedom. Therefore, each of the threetransducers are associated with a degree of freedom of movement of theobject when it is engaged with the object receiving portion of thegimbal mechanism.

More specifically, an apparatus for interfacing an operator manipulableshaft with a computer includes a support, a gimbal mechanism, and foursensors. The gimbal mechanism preferably includes a U shaped baseportion having a base and a pair of substantially parallel legsextending therefrom, where the base of the U shaped base portion isrotatably coupled to the support, and a shaft receiving portionpivotally coupled between the legs of the base portion. The shaftreceiving portion includes a translation interface and a rotationinterface that engage the shaft when it is engaged with an aperture ofthe shaft receiving portion. The base portion rotates around a firstaxis and the shaft receiving portion rotates around a second axissubstantially perpendicular to the first axis, such that an axis of theshaft defines a radius in a spherical coordinate system having an originat an intersection of the first axis and the second axis. A first sensoris coupled between the support and the U shaped base portion to providea first output signal, a second sensor is coupled between the U shapedbase portion and the shaft receiving portion to produce a second outputsignal, a third sensor is coupled to the translation interface toproduce a third output signal, and a fourth sensor is coupled betweenthe rotation interface and the object to produce a fourth output signal.The output signals are preferably coupled to an input of a computer byan electronic interface.

In an alternative embodiment of the present invention a first actuatoris coupled between the support and the U shaped base portion to producea movement therebetween in response to a first input electrical signal,a second actuator is coupled between the U shaped base portion and theshaft receiving portion to produce a movement therebetween in responseto a second input electrical signal, a third actuator is coupled to thetranslation interface to produce a mechanical movement of the elongatedcylindrical object relative to the shaft receiving portion in responseto a third input electrical signal, and a fourth actuator is coupled tothe rotation interface to produce a mechanical movement of the elongatedcylindrical object relative to the shaft receiving portion in responseto a fourth input electrical signal.

A method for providing a human/computer interface includes the steps of:(a) defining an origin in a 3-dimensional space; (b) physicallyconstraining a shaft that can be grasped by an operator such that aportion of the object always intersects the origin and such that theportion of the object extending past the origin defines a radius in aspherical coordinate system; (c) transducing a first electrical signalrelated to a first angular coordinate of the radius in the sphericalcoordinate system with a first transducer; (d) transducing a secondelectrical signal related to a second angular coordinate of the radiusin the spherical coordinate system with a second transducer; (e)transducing a third electrical signal related to the length of theradius with a third transducer; and (f) electrically coupling thetransducers to a computer system to provide a human/computer interface.The method can further include the step of transducing a fourthelectrical signal related to a rotation of the shaft around an axis witha fourth transducer. The transducers are either sensors, actuators, orbi-directional transducers which can serve as either input or sensors.

It will therefore be appreciated that a human/computer interface of thepresent invention includes a support, a gimbal mechanism coupled to thesupport, and an elongated shaft engaged with the gimbal mechanism andhaving a grip area that can be grasped by a hand of an operator. Thegimbal mechanism has a base portion rotatably coupled to the support,and a shaft receiving portion rotatably coupled to the base. A firstsensor is coupled between the support and the base portion, a secondsensor is coupled between the base portion and the shaft receivingportion, and a third sensor is coupled between the shaft receivingportion and an intermediate portion of the shaft. The three sensors arecoupled to an input of a computer to provide the human/computerinterface. Preferably, the interface further includes a fourth sensorcoupled between the shaft receiving portion and an intermediate portionof the shaft, where the third sensor is a translation sensor and thefourth sensor is a rotation sensor.

The advantage of the present invention is that a 3-dimensionalhuman/computer interface tool is provided which has the three or fourdegrees of freedom available that are desirable for many virtual realitysimulation applications. The mechanism of the present invention isrelatively straight-forward allowing for low cost production and highreliability. Furthermore, since the human/computer interface tool of thepresent invention is constrained from movement along at certain degreesof freedom, it can more accurately simulate the use of tools and otherelongated mechanical objects which are similarly constrained.Importantly, the present interface is of low inertia since the primarymass of the interface is located at the pivot point. This, along withthe light weight of the interface, makes the interface less fatiguing touse.

In another embodiment of the present invention a human/computerinterface tool is provided which is provided with only two degrees offreedom. This is particularly advantageous when the shaft is flexible,such as with very thin shafts, wires, catheters, and the like. With, forexample, catheters, it is only necessary to provide two degrees offreedom (i.e. in-and-out, and rotation) and, therefore, sensors and/oractuators for the other degrees of freedom do not need to be provided.

These and other advantages of the present invention will become apparentto those skilled in the art upon a reading of the following descriptionsof the invention and a study of the several figures of the drawing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of a virtual reality system which employsan apparatus of the present invention to interface a laparoscopic toolhandle with a computer system;

FIG. 2 is a perspective view of an apparatus for mechanicallyinterfacing an elongated mechanical object with an electrical system inaccordance with the present invention;

FIG. 2 a is a perspective view of an alternative translation interfaceused for wires, catheters, and the like;

FIG. 3 is front elevation view of the apparatus of FIG. 2 illustrating alaparoscopic tool engaged with an object receiving portion of thepresent invention;

FIG. 4 is a side elevation similarly showing a laparoscopic tool engagedwith the object receiving portion of the present invention;

FIG. 5 is a top plan view also illustrating the engagement of alaparoscopic tool with the object receiving portion of the presentinvention;

FIG. 6 is a pictorial view illustrating the four degrees of freedomenjoyed with the mechanism of the present invention;

FIG. 7 illustrates a first embodiment of an input sensor;

FIG. 8 illustrates a modified laparoscopic tool handle for the use ofthe present invention;

FIG. 8 a is a cross-section taken along line 8 a-8 a of FIG. 8;

FIG. 9 is a perspective view of a sensor in accordance with the presentinvention;

FIG. 9 a is a sectional view taken along line 9 a-9 a of FIG. 9;

FIG. 9 b is a perspective view of an alternative sensing wheel used forwires, catheters, and the like;

FIG. 10 is a perspective view of and alternative sensor mechanism of thepresent invention;

FIG. 10 a is a cross sectional view taken along line 10 a-10 a of FIG.10;

FIG. 11 is a perspective view of another alternative sensor of thepresent invention; and

FIG. 11 a is a sectional view taken along line 11 a-11 a of FIG. 11.

DETAILED DESCRIPTION

In FIG. 1, a virtual reality system 10 includes a human/computerinterface apparatus 12, a electronic interface 14, and a computer 16.The illustrated virtual reality system 10 is directed to a virtualreality simulation of a laparoscopic surgery procedure. The software ofthe simulation is not a part of this invention and thus will not bediscussed in any detail. However, such software is commerciallyavailable as, for example, Teleos™ from High Techsplanations ofRockville, Md. Suitable software drivers which interface such simulationsoftware with computer input/output (I/O) devices are available fromImmersion Human Interface Corporation of Palo Alto, Calif.

A laparoscopic tool 18 used in conjunction with the present invention ismanipulated by an operator and virtual reality images are displayed on ascreen 20 of the digital processing system in response to suchmanipulations. Preferably, the digital processing system is a personalcomputer or workstation, such as an IBM-PC AT or Macintosh personalcomputer, or a SUN or Silicon Graphics workstation. Most commonly, thedigital processing system is a personal computer which operates underthe MS-DOS operating system in conformance with an IBM PC AT standard.

The human/interface apparatus 12 as illustrated herein is used tosimulate a laparoscopic medical procedure. In addition to a standardlaparoscopic tool 18, the human/interface apparatus 12 includes abarrier 22 and a standard laparoscopic trocar 24. The barrier 22 is usedto represent portion of the skin covering the body of a patient. Trocar24 is inserted into the body of the patient to provide an entry andremoval point from the body of the patient for the laparoscopic tool 18,and to allow the manipulation of the laparoscopic tool 18 within thebody of the patient while minimizing tissue damage. Laparoscopic tools18 and trocars 24 are commercially available from sources such as U.S.Surgical of Connecticut. Preferably, the laparoscopic tool 18 ismodified such that the end of the tool (such as any cutting edges) areremoved, leaving only the handle and the shaft. The end of thelaparoscopic tool 18 is not required for the virtual reality simulation,and is removed to prevent any potential damage to persons or property. Agimbal apparatus 25 is shown within the “body” of the patient in phantomlines.

The laparoscopic tool 18 includes a handle or “grip” portion 26 and ashaft portion 28. The shaft portion is an elongated mechanical objectand, in particular, is an elongated cylindrical object. The presentinvention is concerned with tracking the movement of the shaft portion28 in three-dimensional space, where the movement has been constrainedsuch that the shaft portion 28 has only three or four free degrees ofmotion. This is a good simulation of the real use of a laparoscopic tool18 in that once it is inserted into a trocar 24 and through the gimbalapparatus 25, it is limited to about four degrees of freedom. Moreparticularly, the shaft 28 is constrained at some point of along itslength such that it can move with four degrees of freedom within thepatient's body.

While the present invention will be discussed with reference to theshaft portion 28 of laparoscopic tool 18, it will be appreciated that agreat number of other types of objects can be used with the method andapparatus of the present invention. In fact, the present invention canbe used with any elongated mechanical object where is desirable toprovide a human/computer interface with three or four degrees offreedom. Such objects may include catheters, hypodermic needles, wires,fiber optic bundles, screw drivers, pool cues, etc. Furthermore,although the described preferred embodiment of the present inventioncontemplates the use of a elongated cylindrical mechanical object, otherembodiments of the present invention provide a similar human/computerinterface for an elongated mechanical objects which are not cylindricalin shape.

The electronic interface 14 is a part of the human/computer interfaceapparatus 12 and coupled the apparatus 12 to the computer 16. Anelectronic interface 14 that is particularly well adopted for thepresent is described in U.S. patent application Ser. No. 08/092,974,filed Jul. 16, 1993, now U.S. Pat. No. 5,576,727, —assigned to theassignee of the present invention and incorporated herein by referencein its entirety. The electronic interface described therein was designedfor the Immersion PROBE™ 3-D mechanical mouse and has six channelscorresponding to the six degrees of freedom of the Immersion PROBE.However, in the context of the present invention, the electronicinterface 14 requires the use of only four of the six channels, sincethe present invention is preferably constrained to no more than fourdegrees of freedom.

The electronic interface 14 is coupled to a gimbal apparatus 25 of theapparatus 12 by a cable 30 and is coupled to the computer 16 by a cable32. In some embodiments of the present invention, interface 14 servessolely as an input device for the computer 16. In other embodiments ofthe present invention, interface 14 serves solely as an output devicefor the computer 16. In yet other embodiments of the present invention,the interface 14 serves as an input/output (I/O) device for the computer16.

In an alternative embodiment of the present invention, interface 14 hasa local microprocessor 33 preferably coupled with any transducerspresent in the interface 14 and with a transceiver 35. In such anembodiment, the computer 16 is coupled to the transceiver 35 and,typically, not coupled directly with any transducers present in theinterface 14. As will be appreciated, the transceiver 35 may be anysuitable transceiver capable of bi-directional communication throughserial or parallel communication strategies. The local microprocessor 33will be programmed to execute computer instructions locally such that acomputing burden is removed from the computer 16. For example,positional information generated by the transducers may be processedlocally by the local microprocessor 33, which in turn can send absoluteposition and velocity information to the computer 16. Still further, thelocal microprocessor 33 is capable of receiving incoming force commandsfrom the computer 16, decoding such commands, and controlling theinterface 14 accordingly. For more details, see U.S. Pat. No. 5,576,727of Rosenberg et al.”

In the perspective view of FIG. 2, the gimbal apparatus 25 of thepresent invention is illustrated in some detail. The gimbal apparatus 25includes a support 34 and a gimbal mechanism 36 rotatably coupled to thesupport. The gimbal mechanism 36 preferably includes a U shaped baseportion 38 including a base 40 and a pair of substantially parallel legs42 a and 42 b extending upwardly therefrom. As used herein,“substantially parallel” will mean that two objects or axis are exactlyor almost parallel, i.e. are at least within five or ten degrees ofparallel, and are preferably within less than one degree of parallel.Similarly, the term “substantially perpendicular” will mean that twoobjects or axis are exactly or almost perpendicular, i.e. at leastwithin five degrees or ten degrees of perpendicular, or more preferablywithin less than one degree of perpendicular.

The gimbal mechanism 36 also includes an elongated object (shaft)receiving portion 44 provided with an aperture 46 which extends entirelythrough the object receiving portion. The aperture 46 defines an objectaxis A₀ for an elongated cylindrical object, such that the shaft portion28 of the laparoscopic tool 18 of FIG. 1. The object receiving portion44 is at least partially disposed between the legs 42 a and 42 b of theU shaped base portion, and is pivotally coupled thereto such as by apair of pivots, one of which is shown as pivot 48 a in leg 42 a. Anotherpivot 48 b (not shown) is provided in leg 42 b.

The object receiving portion 44 also includes a translation interface 50and a rotation interface 52. The object receiving portion 44 includes abearing section 54, a translation sensor section 56, and a rotationsensor section 58. The bearing section 54 includes a mass of materialprovided with a cylindrical bore 60 forming a portion of the aperture46. The translation sensor section 56 includes a pair of opposing wallsurfaces 62 a and 62 b, each of which is provided with a cylindricalbore receptive to the cylindrical object and forming a part of theaperture 46 which extends through the object receiving portion. Thetranslation sensor section 56 includes a pair of opposing wall surfaces64 a and 64 b of a wall 63 and which are provided with cylindrical boresreceptive to the cylindrical object and therefore also forming a part ofthe aperture 46. In consequence, when an elongated cylindrical object isinserted into the object receiving portion 44 along axis A₀ it engagesthe bore 60 of the bearing section 54, and extends through boresprovided in the surfaces 62 a, 62 b, 64 a, and 64 b to extend completelythrough the object receiving portion 44 along aperture 46. In anotherembodiment of the present invention, wall 63 (and therefore wallsurfaces 64 a and 64 b) is eliminated as being superfluous.

Referring briefly to FIG. 2 a, an alternative construction for thetranslation interface 50 of FIG. 2 is shown at 50′. This alternativetranslation interface 50′ is well adapted for very thin shafts, wires,catheters, and the like. The problem encountered with the translationinterface 50 is that, for example, wires and catheters are flexible andtherefore do not engage well with a single friction wheel. Therefore,the translation interface 50′ includes a drive wheel 65 a that iscoupled to a sensor and/or actuator, and an idler wheel 65 b. The wireor catheter 67 is pinched between the drive wheel 65 a and the idlerwheel 65 b so that there is good frictional engagement between thecatheter 67 and the drive wheel 65 a.

The object receiving portion 44 is preferably a unitary mass of materialmade from aluminum or some other lightweight material such as a plastic.The object receiving portion 44 is preferably cast, molded, and/ormachined as a monoblock member having the aforementioned bearingsection, translation sensory section, and rotation sensory section. Thematerials and construction of U shaped base portion 38 preferably matchthe materials and construction techniques used for the production ofobject receiving portion 44.

The gimbal apparatus 25 illustrated in FIG. 2 constrains an object thatis engaged with the object receiving portion 44 to four degrees offreedom. This is accomplished by allowing the U shaped base portion 38to rotate around an axis A₁ relative to the support 34, by allowing theobject receiving portion 44 to rotate around an axis A₂ relative to theU shaped base portion 38, by allowing the object to translate asillustrated by the arrow t along axis A₀ of aperture 46, and by allowingthe object to rotate as indicated by arrow r around the axis A₀ ofaperture 46.

Four electromechanical transducers are used in association with thesefour degrees of freedom. More particularly, a first degree of freedomelectromechanical transducer 66 is arranged to transduce motion and/orforce between the U shaped base portion 38 and the support 34, a seconddegree of freedom electromechanical transducer 68 is arranged totransduce motion and/or force between the U shaped base portion 38 andthe object receiving portion 44, a third degree of freedomelectromechanical transducer 70 is arranged to transduce motion and/orforce between the object receiving portion 44 and an object engaged withthe object receiving portion 44, and a fourth degree of freedomtransducer 72 is arranged to transduce motion and/or force between theobject receiving portion 44 and an object engaged with the objectreceiving portion 44.

By “associated with”, “related to”, or the like, it is meant that theelectromechanical transducer is influenced by or influences one of thefour degrees of freedom. The electromechanical transducers can be inputtransducers, in which case they sense motion along a respective degreeof freedom and produce an electrical signal corresponding thereto forinput into computer 16. Alternatively, the electromechanical transducerscan be output transducers which receive electrical signals from computer16 that cause the transducers to impart a force on the object inaccordance with their respective degrees of freedom. Theelectromechanical transducers can also be hybrid or bi-directionaltransducers which operate both as sensors and as actuator devices.

A variety of transducers, readily available in the commercial market aresuitable for use in the present invention. For example, if thetransducers are input transducers (“sensors”), such sensors can includeencoded wheel transducers, potentiometers, etc. Output transducers(“actuators”) include stepper motors, servo motors, magnetic particlebrakes, friction brakes, pneumatic actuators, etc. Hybrid orbi-directional transducers often pair input and output transducerstogether, but may also include a purely bi-directional transducer suchas a permanent magnet electric motor/generator.

It should be noted that the present invention can utilize both absoluteand relative sensors. An absolute sensor is one which the angle of thesensor is known in absolute terms, such as with an analog potentiometer.Relative sensors only provide relative angle information, and thusrequire some form of calibration step which provide a reference positionfor the relative angle information. The sensors described herein areprimarily relative sensors. In consequence, there is an impliedcalibration step after system power-up wherein the shaft is placed in aknown position within the gimbal mechanism and a calibration signal isprovided to the system to provide the reference position mentionedabove. All angles provided by the sensors are thereafter relative tothat reference position. Such calibration methods are well known tothose skilled in the art and, therefore, will not be discussed in anygreat detail herein.

A preferred input transducer for use of the present invention is anoptical encoder model SI marketed by U.S. Digital of Vancouver, Wash.This transducer is an encoded wheel type input transducer. A preferredoutput transducer for use of the present invention is a d.c. motor model2434.970-50 produced by Maxon of Fall River, Mass. This type oftransducer is a servo motor type output transducer.

There are a number of ways of attaching the transducers to the variousmembers of the gimbal apparatus 25. In this preferred embodiment, ahousing of transducer 66 is attached to the U shaped base portion 38,and a shaft of the transducer extends through an oversize bore (notshown) in base 40 to engage a press-fit bore (also not shown) in support34. Therefore, rotation of the us shaped base portion 38 around axis A₁will cause a rotation of a shaft of transducer 66. A housing oftransducer 68 is attached to leg 42 a of the U shaped base portion 38such that its shaft forms pivot 48 a. Therefore rotation of the objectreceiving portion 44 around axis A₂ will cause a rotation of the shaftof a second transducer 68. The transducer 70 is attached to objectreceiving portion 44 and extends through a bore (not shown) in a wall 74of the translation sensor section 56. The shaft 76 provides an axisabout which the translation interface 50 can rotate. The fourthtransducer 74 is attached to a wall 78 of rotation sensor section 58 andextends through a bore 80 in that wall 78. The shaft 82 of thetransducer 72 engages a circumferential surface of rotation interface 52and rotates therewith.

Axes A₁ and A₂ are substantially mutually perpendicular and intersect atan origin point O within object receiving portion 44. Axis A₀ alsointersects this origin O. Shaft 76 rotates around an axis A₃ which issubstantially perpendicular to the axis A₀. Shaft 58 of transducer 72rotates around an axis A₄ which is substantially parallel to the axisA₀.

In FIG. 3, a front view of the gimbal apparatus 25 is used to illustrateone of the degrees of motion of the laparoscopic tool 18. Theillustrated degree of freedom is the fourth degree of freedom, i.e.rotation around axis A₀ as illustrated by the arrow r in FIG. 2. Thisdegree of freedom is detected by transducer 72. In this fourth degree ofmotion, the handle portion 26 of the laparoscopic tool 18 can rotate ina clockwise direction as indicated at 26′ and in a counter clockwisedirection as indicated at 26″. Of course, the handle 26 can rotate afull 360° although this would require the release and re-grasping of thehandle 26.

In FIG. 4, a second degree of freedom is illustrated. With this degreeof freedom, the laparoscopic tool 18 can pivot upwardly as illustratedat 18′ or downwardly (not shown). This rotation around A₂ is detected bytransducer 68. It should be noted in the present embodiment, thelaparoscopic tool 18 cannot rotate 360° around the axis A₂ because it isphysically constrained by the support 34, portions of the gimbalmechanism 36, etc. However, in the present embodiment, the laparoscopictool can achieve approximately 170 degrees of rotation around axis A₂.

FIG. 5 is top view of the gimbal apparatus 25 and illustrates the firstand third degrees of freedom. The first degree of freedom is detected bytransducer 66 as the laparoscopic tool 18 is pivoted or rotated aroundaxis A₁ as illustrated at 18 a and 18 b. The third degree of freedom isdetected by transducer 70 as the shaft portion 28 of laparoscopic tool18 is moved back and fourth as illustrated by the arrow “t.” This causesa rotation of translation interface 50 and the shaft 76 of the thirdtransducer 70.

The four degrees of freedom are illustrated graphically in FIG. 6. Thecylinder 66′ represents the first transducer 66 and allows a firstdegree of freedom labeled “1st” around axis A₁. Cylinder 68′ representsthe sensor 68 and allows a second degree of freedom labeled “2nd” aroundaxis A₂. Telescoping members 70 a′ and 70 b′ represent the third sensor70 can sense movement along a third degree of freedom labeled “3rd”along axis A₀. Finally, a cylinder 72′ attached to member 70 b′represents the fourth transducer 72 and senses a fourth degree offreedom labeled “4th” around axis A₀. A member 84 is provided toindicate position and rotational direction relative to axis A₀.

In FIG. 7, a preferred input transducer (sensor) of the presentinvention is disclosed. Again, an input transducer of this type can bepurchased as sensor model SI from U.S. Digital of Vancouver, Wash. Theinput transducer 86 includes a bearing block 88 having a bearing 89, arotary shaft 90 supported by the bearing 89, and a sensing wheel 92supported for rotation by shaft 90. The sensing wheel is preferably madefrom a clear, plastic material and is provided with a number of darkradial bands 94 near its circumference, such as by printing or silkscreening. A first photodetector pair 96 a including a light source 98 aand a detector 100 a are positioned on opposing sides of the sensingwheel 92 in alignment with the bands 94. Similarly, a secondphotodetector pair 96 b including a light source 98 b and a detector 100b are positioned on opposing sides of the sensing wheel 92 in alignmentwith the bands 94. As the sensing wheel 92 rotates as illustrated at 102around an axis A, the bands 94 alternatively allow light emanating fromlight sources 98 a and 98 b to impinge or not impinge upon the detectors100 a and 100 b, respectively. The electronic interface 14, coupled tothe photodetector pairs 96 a and 96 b by cable 30, counts the bands 94as they pass the photodetector pairs 96 a and 96 b to provide a signalon cable 32 to the computer 16 indicating the rotational position of theshaft 90 around axis A. The two pairs 96 a and 96 b are provided todetermine the direction of rotation, as is well known to those skilledin the art of sensor design.

FIGS. 8 and 8 a illustrate a modified laparoscopic tool 104. Moreparticularly, a sensor 106 has been added to determine when the handle108 has been squeezed, and the shaft 110 has been grooved or slotted fora purpose to be discussed subsequently. The sensor 106 can be coupled tothe computer 16 through electronic interface 14 to provide additionalinput to the virtual reality system.

With reference to FIG. 8 a, the shaft 110 is preferably hollow, havingan axial bore 112 which aligns with axis A₀, and is provided with anelongated groove 114 which is parallel to an axis A_(L) of the shaft110. This elongated groove 114 can be produced by any process includingextruding the shaft 110 in the appropriate shape, or cutting the groove114 with a machine tool, etc.

FIGS. 9 and 9 a illustrate an alternate embodiment for transducer 72which utilizes the shaft 110 and a detector mechanism similar to the oneillustrated in FIG. 7. More particularly, the transducer 72′ includes asleeve 114 which is slidingly engaged with shaft 110. As seen in thecross sectional view of FIG. 9 a, the sleeve 115 is a substantiallycylindrical object having a central bore 116 which engages thecircumference 118 of the shaft 110. The sleeve 115 has a key 120 whichengages the groove 114 of the shaft 110. Therefore, while the sleeve canslide back and forth along the axis A_(L) as indicated at 122, but thesleeve 115 rotates with the shaft 110 as indicated at 124 due to theengagement of the key 120 with the groove 114. A sensing wheel 92′ isaffixed to a circumferential portion of sleeve 115 so that it rotatescoaxially with the sleeve 115. A photodetector pair 96′ senses themotion of bands 94′ and produces an electrical signal on cable 30. Theadvantage of the embodiment shown in FIGS. 9 and 9 a is that rotation ofthe shaft around axis A_(L) is detected without the possibility ofslippage. Another advantage of this embodiment is that it is morecompact in design.

In FIG. 9 b an alternate embodiment for a rotation interface 52′ isshown. This alternate embodiment is well adapted for flexible shafts,wires, catheters and the like, such as the aforementioned catheter 67.The rotation interface 52′ includes a transducer 72″″ that is providedwith a resilient grommet 73 having a hole that engages a circumferentialportion of the catheter 67. The grommet 73 is preferably a rubber orplastic grommet that causes the catheter 67 to rotate coaxially as thecatheter spins or rotates. Preferably, the mass of the transducer 72″″is kept very small so that it only takes a small amount of friction toensure coaxial rotation of the catheter and transducer without slippage.Because the level of friction is so small, it does not substantiallyimpede translational motion (i.e. in-out motion) of the catheter.

FIGS. 10 and 10 a illustrate another embodiment 72″ for the transducer72 of FIG. 2 This embodiment has a number of points of similarity withthe embodiment discussed with reference to FIGS. 9 and 9 a, and it willbe appreciated that elements with like reference numerals operate in asimilar fashion. However, the embodiment of FIGS. 10 and 10 a include asheave 126 affixed to the circumference of sleeve 115 in the place ofthe sensing wheel 92′ of FIG. 9 and FIG. 9 a. A position sensor 128 hasa shaft 130 which is coupled to the sheave 126 by a belt 132. The belt132 can be any continuous loop structure including a resilient,rubber-type belt, a drive-chain type belt, etc. The shaft 130 ofposition sensor 128 therefore rotates with the sheave 126. The advantageof using a belt 132 or the like is that a substantial amount of forcemay be applied to the belt to, again, minimize slippage.

Another embodiment 72′″ for the fourth transducer is illustrated inFIGS. 11 and 11 a. Again, there are a number of points of similaritybetween the embodiments of FIGS. 11 and 11 a and the previouslydescribed embodiments of FIGS. 9 and 9 a and FIGS. 10 and 10 a.Therefore, like reference numerals will again refer to like elements. Inthis embodiment, a sensor 134 haws a shaft 136 which serves as the axleof a friction wheel 138 which, in turn, engages a circumferentialsurface of sleeve 115. Therefore, a rotation of the shaft 110 will causea rotation of the sleeve 115, which will cause a rotation of the wheel138 and the shaft 136 to create an electrical signal on cable 30.

With reference to all of the figures, and with particular reference toFIGS. 1 and 2, the shaft 28 of a laparoscopic tool 18 is inserted intoaperture 46 along axis A₀, causing the shaft 28 to frictionally engagethe translation interface (wheel) 50. In this instance, thetranslational interface 50 is a friction wheel made out of a rubber-likematerial. The shaft 28 is also in engagement with the rotationalinterface 52 which, in the embodiment of FIG. 2, is also a frictionalwheel made out of a rubber-like material. Rotation of the shaft 28around the axis A₀ is illustrated by the arrow r will cause a rotationof the friction wheel 50 and therefore the shaft 82 of the sensor 72. Atranslation of the shaft 28 along with axis A₀ will cause a rotation ofthe friction wheel 50 which rotates the shaft 76 of the transducer 70. Amovement up or down of the laparoscopic tool 18 will cause a rotation ofthe shaft (pivot) 48 a of transducer 68, and a side-to-side pivoting ofthe laparoscopic tool 18 will cause a rotational around axis A₁ which isdetected by transducer 66.

To this point, the majority of the discussion has been under theassumption that the transducers are input transducers, i.e. thehuman/computer interface device is used an input device to the computer16. However, it is also been mentioned that the interface device 12 canserve as an output device for the computer 16. When used as an outputdevice, output transducers (“actuators”) are used to respond toelectrical signals developed by the computer 16 to impart a force uponthe shaft 28 of the laparoscopic tool 18. This can provide usefulmovement and force (haptic) feedback to the doctor/trainee or otheruser. For example, if the laparoscopic tool encounters dense mass oftissue or a bone in the “virtual” patient, a force can be generated bytransducer 70 making it harder for the doctor/trainee to push the shaft28 further into the gimbal apparatus 25. Likewise, twisting motions canbe imparted on the shaft 28 when the shaft encounters an obstacle withinthe virtual patient.

It should be noted that force applied to the shaft may not result in anymovement of the shaft. This is because the shaft may be inhibited frommovement by the hand of the operator who is grasping a handle or gripportion of the shaft. However, the force applied to the shaft may besensed by the operator as haptic feedback.

With reference to FIG. 2, a method for mechanically interfacing anelongated mechanical object with an electrical system in accordance withthe present invention includes first step of defining an origin in3-dimensional space. This corresponds to the origin O at theintersection of axis A₁ and A₂. A second step is to physically constrainan elongated object in the 3-dimensional space such that a portion ofthe object always intersects the origin O and such that a portion of theobject extending from the origin O defines a radius in a sphericalcoordinate system. The elongated object (such as shaft 28 oflaparoscopic tool 18) is physically constrained in a 3-dimensional spaceby the aperture 46 of the object receiving portion 44. The portion ofthe shaft 28 extending from origin O defines the radius. A third stepincludes transducing a first electrical signal related to a firstangular coordinate of the radius with a first transducer. Thiscorresponds to the operation of transducer 66 which transduces a firstelectrical signal related to a first angular coordinate of the radius. Afourth step is transducing a second electrical signal related to asecond angular coordinate of the radius. This corresponds to theoperation of transducer 68 which transduces a second electrical signal.A fifth step is to transduce a third electrical signal related to thelength of the radius, which corresponds to the operation of transducer70. A sixth and final step is to electrically couple the transducers toan electrical system which, in this instance, is preferably a computer16. An additional, optional step transduces a fourth electrical signalrelated to a rotation of the object around an object axis whichintersects the origin O. This step corresponds to the operation oftransducer 72. The transducers can be input transducers, outputtransducers, or bi-directional transducers.

It will be noted that the electrical system most frequently described inthe present invention is a digital processing system or a computer.However, other digital systems, analog systems, and simple electric orelectromechanical system can also be utilized with the apparatus andmethod of the present invention.

It will also be noted that while specific examples of “elongatedobjects” and “shafts” have been given, that these examples are not meantto be limiting. In general, equivalents of “elongated objects”,“elongated cylindrical objects”, “shafts”, etc. include any object whichcan be grasped by a human operator to provide an interface between theoperator and a computer system. By “grasp”, it is meant that operatorsmay releasably engage a grip portion of the object in some fashion, suchas by hand, with their fingertips, or even orally in the case ofhandicapped persons. The “grip” can be a functional grip or handleattached to an elongated portion of the object, or can be a portion ofthe object itself, such as a portion of the length of a shaft that canbe gripped and/or manipulated by the operator.

It should also be noted that flexible shafts, such as wires orcatheters, do not always require three or four degrees of freedom. Forexample, if a human/computer interface for a catheter insertion virtualreality system is desired, only a translation interface (e.g.translation interface 50′ of FIG. 2 a) and rotation interface (such asrotation interface 52′ of FIG. 9 c) may be required. This is because acatheter can be moved in and out of a virtual patient (as sensed bytranslation interface 50′) and can be twisted or rotated (as sensed byrotation interface 50′), but cannot be, in any practical manner, movedup or down or from side-to-side due to the flexibility of the catheter.In such applications, therefore, it is desirable to have ahuman/computer interface with only two degrees of freedom.

While this invention has been described in terms of several preferredembodiments, it is contemplated that alternatives, modifications,permutations and equivalents thereof will become apparent to thoseskilled in the art upon a reading of the specification and study of thedrawings. It is therefore intended that the following appended claimsinclude all such alternatives, modifications, permutations andequivalents as fall within the true spirit and scope of the presentinvention.

1. A method, comprising: generating a graphical simulation on a computercoupled to a display; manipulating a user controlled object coupled toan interface including a mechanism, the mechanism configured to allowmovement of the user controlled object in at least three rotary degreesof freedom all intersecting at a single common pivot point, the commonpivot point between a first end and a second end of the user controlledobject, wherein the first end of the user controlled object is contactedby a user; sensing movement of the user controlled object in a lineardegree of freedom as the user controlled object is slidably movedthrough the mechanism of the interface; transmitting positioninformation of the sensed movement to the computer; updating thedisplayed graphical simulation in response to the updated positioninformation; transmitting a force command from the computer to theinterface based on the updated position information; and generating aforce on the user controlled object in response to the force command. 2.The method of claim 1, wherein the generating the graphical simulationfurther comprises generating a graphical representation of a body on thedisplay.
 3. The method of claim 1, wherein the generating the graphicalsimulation further comprises generating a graphical representation ofthe user controlled object on the display.
 4. The method of claim 1,wherein the generating the force further comprises generating aresistance force on the user controlled object to simulate a pushingsensation, wherein the resistance force is in the linear degree offreedom.
 5. The method of claim 1, wherein the generating the forcefurther comprises generating a resistance force on the user controlledobject about a rotary degree of freedom.
 6. The method of claim 1,wherein the generating the graphical simulation further comprisesgenerating a graphical representation of the user controlled object onthe display, wherein the force command is transmitted upon andcorresponds to the graphical representation of the user controlledobject coming into contact with an obstacle in the graphical simulation.7. The method of claim 1, wherein the user controlled object alwaysintersects the common point and a portion of the user controlled objectextending from the common point defines a radius of a sphere.
 8. Themethod of claim 1, wherein said sensing movement further comprises:sensing movement of the user controlled object about a first axis; andsensing movement of the user controlled object about a second axis,wherein the first and second axes are substantially perpendicular to oneanother.
 9. A method comprising: generating a graphical simulation on acomputer coupled to a display; sensing movement of a user controlledobject coupled to an interface having a mechanism, wherein the mechanismis configured to allow movement of the user controlled object in atleast three rotary degrees of freedom all intersecting at a singlecommon pivot point, the common pivot point between a first end and asecond end of the user controlled object, wherein the first end of theuser controlled object is contacted by a user, wherein the sensedmovement is in a rotary degree of freedom; transmitting positioninformation of the sensed movement to the computer, wherein the sensedlinear movement is of the user controlled object sliding through themechanism of the interface; updating the displayed graphical simulationin response to the updated position information; transmitting a forcecommand from the computer to the interface based on the updated positioninformation; and generating a force on the user controlled object to befelt by the user in response to the force command.
 10. The method ofclaim 9, wherein said generating the graphical simulation furthercomprises generating a graphical representation of a body on thedisplay.
 11. The method of claim 9, wherein said generating thegraphical simulation further comprises generating a graphicalrepresentation of the user controlled object on the display.
 12. Themethod of claim 9, wherein said generating the force further comprisesgenerating a resistance force on the user controlled object to simulatea pushing sensation, wherein the resistance force is in a linear degreeof freedom.
 13. The method of claim 9, wherein said generating the forcefurther comprises generating a resistance force on the user controlledobject about a rotary degree of freedom.
 14. The method of claim 9,wherein the generating the graphical simulation further comprisesgenerating a graphical representation of the user controlled object onthe display, wherein the force command is transmitted upon andcorresponds to the graphical representation of the user controlledobject coming into contact with an obstacle in the graphical simulation.15. The method of claim 9, wherein the user controlled object alwaysintersects the common point and a portion of the user controlled objectextending from the common point defines a radius of a sphere.
 16. Themethod of claim 9, wherein said sensing movement further comprises:sensing movement of the user controlled object about a first axis; andsensing movement of the user controlled object about a second axis,wherein the first and second axes are substantially perpendicular to oneanother.
 17. An interface apparatus comprising: means for engaging auser controlled object having a first end and a second end, the firstend in contact with and physically manipulatable by a user, wherein themeans for engaging is configured to allow movement of the usercontrolled object in at least three rotary degrees of freedom allintersecting at a single common pivot point, the common pivot pointbetween the first end and the second end of the user controlled object;means for sensing movement of the user controlled object in at least onedegree of freedom when the user controlled object is slidably moved in alinear direction through the means for engaging; means for outputtingdata indicative of the linear movement to a computer running a graphicaluser interface program, wherein the computer is configured to update thegraphical user interface program in response to receiving the data; andmeans for outputting a force to the user controlled object to be felt bythe user in response to receiving a control signal from the computersystem, wherein the force is applied in the degree of freedom.
 18. Theinterface apparatus of claim 17, wherein said means for outputting theforce generates a resistance force against the movement of the usercontrolled object in a linear degree of freedom.
 19. The interfaceapparatus of claim 17, wherein said means for outputting the forcegenerates a resistance force against the movement of the user controlledobject in a rotary degree of freedom.